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DELAYED STIMULATION IN AUDITORY PROSTHESES 

The present invention relates generally to the generation of electrical stimuli 
for application by an auditory prosthesis. The invention is suitable for use in the 
stimulation of cochleas or the auditory brain and it will be convenient to describe 
5 the invention in relation to these exemplary, non-limiting applications. 

Early signal processing designs for the multi-channel cochlear implant 
extracted the second formant (F2) and pitch (F0) to control electrode stimulation. 
The frequency of F2 controlled the location of electrode stimulation, and F0 
controlled the rate of stimulation with only one electrode stimulated per pitch 

10 period. Improvements were made by also extracting the first formant (F1) and 
adding a corresponding second stimulated electrode for each pitch period. The 
MULTIPEAK stimulation strategy, described in US Patent No. 5,271,397, added 
stimulation of three fixed electrodes to add high-frequency information. 
Stimulation times were still controlled by F0 for voiced speech or were random for 

15 unvoiced speech. 

The Spectral Maxima Sound Processor (SMSP) strategy, described in 
Australian Patent No. 657,959, and SPEAK strategy, described in US Patent No. 
5,597,380, were a departure from the others as they used a fixed stimulation rate 
and stimulated electrodes that corresponded to maxima in the sound spectra. 

20 Another fixed-rate strategy, CIS, is described in US Patent No. 4,207,441. This 
strategy stimulated all of a small number of electrodes to represent the sound 
spectra. More recently the ACE strategy was developed which is able to perform 
all of the above strategies. Several stimulation orderings were investigated for the 
SMSP strategy (Vandali et al. (1995) "Multichannel cochlear implant speech 

25 processing: Further variations of the spectral maxima sound processor strategy", 
Annals of Otology, Rhinology & Laryngology, Supplement 166, Vol. 104, No. 9, 
Part 2, pp. 378-381): amplitude order, including largest-to-smallest and smallest- 
- to-largest, where the stimulation order for each analysis cycle was controlled by 
the amplitudes of the maxima; and tonotopic order, where stimulation order was 
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from highest to lowest frequency. The tonotopic ordering scheme showed a small 
improvement for speech perception in noise. The SPEAK and ACE strategies, by 
default, all use the tonotopic ordering scheme. 

Other recent developments are "A peak-derived timing stimulation strategy 
5 for a multi-channel cochlear implant" described in International Patent Application 
No. WO 02/096153, "Sound processor for a cochlear implant" described in 
International Patent Application No. WO 01/99470 (called the "Travelling Wave 
Strategy") and "Generation of electrical stimuli for application to a cochlea" 
described in International Patent Application No. PCT/AU03/00639 (called 
10 "STAR"). These strategies use filters to extract spatio-temporal information about 
the incoming audio signal and then stimulate the auditory nerve at times based on 
the properties of the filtered signals. The Travelling Wave Strategy and STAR also 
introduce travelling wave delays to control timing of excitation. 

The auditory brainstem implant is an alternative auditory prosthesis for 
15 people and is usually implanted in the cochlear nucleus. An auditory brain implant 
is a device for stimulation of any area of the auditory system, including the inferior 
colliculus (midbrain) and auditory cortex. These are generally constructed as a 
cluster of electrodes in a grid pattern rather than the linear construction of cochlear 
electrodes. The electrodes are surgically placed in the auditory brain and are 
20 stimulated using similar stimulation strategies as cochlear implants, after 
determining the "place-pitch" equivalents of the implanted electrodes so that 
assignment of frequencies may be made to the electrodes. 

It would be desirable to provide a method and system for generating stimuli 
for application by an auditory prosthesis array that results in the response of 
25 auditory brain neurons more closely resembling the response of a normal hearing 
listener. 

It would also be desirable to provide a method and system for generating 
stimuli for application by an auditory prosthesis that ameliorates or overcomes one 
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or more disadvantages of known electrical stimuli generation methods and 
systems. 

One aspect of the invention provides a method of generating stimuli by an 
auditory prosthesis, including an array of stimulation devices, in response to an 
5 incoming acoustic signal, the method including: 

determining stimulation devices to be activated within the array and 
activation times for those devices; and 

applying a temporal adjustment to the activation times such that activation 
of devices representing lower-amplitude components of the signal is delayed 
10 relative to activation of a proximate device representing a higher-amplitude 
component of the signal. 

The auditory prosthesis may be implantable in a cochlea and forms a linear 
array. Alternatively, the auditory prosthesis may be implantable in an auditory 
brain and form a grid mapped to the form of a linear array. 

15 The temporal adjustment of the activation times may be determined using 

delays derived in a manner similar to lateral suppression of amplitude. In the 
latter, the amplitude of a particular frequency component is reduced by an amount 
determined by the amplitudes of surrounding components. In the temporal 
adjustment scheme, the amplitudes of surrounding components instead introduce 

20 delays in stimulation of the particular frequency component. This is termed "lateral 
temporal delay". 

The activation time of each stimulation device may be temporally adjusted 
according to a latency function whereby, for a particular device, a temporal 
adjustment is applied if the weighted sum of the amplitudes of proximate stimuli 
25 exceeds the weighted amplitude of the stimuli be applied by the particular 
electrode. 
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In one embodiment, the latency function defines a Mexican-hat shape 
centred on the stimuli to be applied by the proximate device, with the restriction of 
being limited to a minimum of no delay. 

The latency function f x (x) may be defined by: 

N 

5 f x (x) = min(0 ,-2aA x + a£ g(y)A y ) 

y=l 
y*x 

where A x is the amplitude of a stimulation to be applied by device x, a is a 
scaling factor, N is the number of devices to which the latency function is 
constrained, and g(y) is amplitude of a stimuli applied by device y . 

Where the auditory prostheses requires non-simultaneous stimulation to be 
1 0 applied by the array of stimulation devices, the method may further include: 

if there is temporal contention between stimulation to be applied by 
different devices of the array, discarding one or more lower-amplitude stimuli in 
favour of a higher-amplitude stimulus. 

Where the auditory prostheses requires non-simultaneous stimulation to be 
1 5 applied by the array of stimulation devices, the method may further include: 

if there is temporal contention between stimulation to be applied by different 
devices of the array, applying a further temporal delay to one or more lower- 
amplitude stimuli by one or more stimulation slots in favour of a higher-amplitude 
stimulus. 

20 In one embodiment of the invention, the array of stimulation devices 

includes one or more electrodes, each electrode being activated by the application 
of a stimulation pulse. 
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In another embodiment, the array of stimulation devices includes one or 
more drug delivery units for the delivery of drugs to a user at predetermined 
locations. The drug delivery units may be realised as fluidic microchannels. 

Another aspect of the invention provides a system for generating stimuli for 
5 application by an auditory prosthesis including an array of stimulation devices, the 
system including: 

a stimulator unit for selectively activating stimulation devices in the array; 

and 

a processor for processing received sound signals and controlling the 
10 operation of the stimulator unit by carrying out a method as described above. 

In embodiments of the invention where the stimulation devices are 
electrodes, the stimulator unit may act to activate the one or more electrodes by 
selectively applying stimulation pulses to the electrodes. 

In embodiments of the invention where the stimulation devices are drug 
15 delivery units, the stimulator unit may include a drug storage device and a drug 
delivery pump for delivering drugs stored in the drug storage device through the 
drug delivery units to a user. 

A further aspect of the invention provides a processor for use in a system 
for generating stimuli for application by an auditory prosthesis including an array of 
20 stimulation devices, the system including a stimulator unit for selectively activating 
stimulation devices in the array, the processor including digital signal processing 
means for processing received sound signals and controlling the operation of the 
stimulator unit by carrying out a method as described above. 

Neurophysiological recordings suggest that the cochlear nucleus, the first 
25 stage of auditory processing in the brainstem, converts frequency information to 
timing information in a dynamic fashion where frequencies of interest are 
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processed faster than less relevant information. This is achieved through 
interactions of inhibition and excitation. The frequencies of interest are those 
whose amplitude is greater than surrounding frequencies. The present invention 
generates stimuli for application to a cochlea, auditory brainstem or other region of 
5 the auditory brain via an auditory prosthesis where the timing of stimulation can be 
modified by a latency model based on physiological data obtained from extra- and 
intra-cellular recordings in the ventral cochlear nuclei. The excitation of cochlear 
implant or auditory brain implant stimulation devices is delayed for lower-amplitude 
frequency bands relative to their neighbouring higher-amplitude frequency bands. 

10 The general sound processing strategy to which the invention can be 

applied may be any strategy currently implemented or proposed for cochlear 
implant or auditory brain implant stimulation. These strategies will hitherto be 
referred to as the "base" strategies. In particular, strategies that stimulate 
electrodes at precise times based on the properties of the incoming acoustic signal 

15 would be preferred. These include, but are not limited to, the Peak-Derived Timing 
Stimulation strategy, the Travelling Wave strategy and the STAR strategy. 

The invention provides a method for processing the stimulation sequences 
resulting from existing sound processing strategies to generate stimuli for 
application by an auditory prosthesis including an array of stimulation devices, the 
20 method entailing the introduction of delay in stimulation time for devices depending 
on their stimulation amplitude compared to the stimulation amplitude proximate 
devices. The time of activation of a stimulation device is obtained from the time 
normally used by the base stimulation strategy and a latency function. 

Physiological data show that if a delay is introduced (irrespective of that 
25 already introduced to compensate for travelling wave) to frequencies which are not 
important then information of importance such as formants in speech or signals in 
noise will be sent to brain sooner and this will aid in their identification and improve 
speech recognition. This delay may be particularly important for speech 
perception in noise as the neural elements involved in this processing respond 
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best to noise. A mechanism also exists to enhance this delay further through 
bilateral inhibitory connections between cochlear nuclei. 

The ventral cochlear nucleus (VCN) stellate population has been divided 
into T and D Stellate cells. D Stellate cells are inhibitory and known to project to T 
5 Stellate neurons, which are excitatory. Intracellular in vivo studies in the VCN 
demonstrate that D Stellate neurons display a significantly shorter latency to initial 
depolarisation than T Stellate cells, which is also reflected in shorter first-spike 
latency. Whereas prolonged inhibition can reduce spike regularity in T stellate 
neurons, given the morphological organisation of D and T stellate cells, the data 

10 provides compelling evidence that, for a tone at a given frequency, fast duration 
inhibitory input from D Stellate cells may delay the onset of firing of T Stellate cells 
with CF's below or above that of the tone depending on its intensity. This implies 
that the timing of action potential generation may be related to the frequency of 
presentation. For a given frequency then, D stellate cells delay the firing of 

15 neighbouring T stellate cells located in different iso-frequency laminae providing a 
timing que for frequency identification. 

The D Stellate cells' inhibition is fast both in duration (<10 ms) and synaptic 
delay (-0.3 ms) and is most responsive to broadband stimuli. It may play a crucial 
role in establishing appropriate neural delays without the need for anatomically 

20 arranged delay lines. As inhibition via D stellate cells is more likely to be activated 
in noise and if, as proposed, inhibition plays a role in maintaining regular chopping 
behaviour in T Stellate cells, the timing and interplay of excitation and inhibition 
may be particularly crucial for signal detection in noisy environments. This has 
implications not only for signal detection in noise but also for coding as a whole 

25 and may circumvent the need for and provide a more dynamic organisation than 
anatomically arranged delay lines in the coding of monaural and binaural 
information. 

In a normal hearing person's auditory pathway, the inhibitory connections 
may be performing this role. However, auditory brain implants bypass the auditory 
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nerve and thereby remove the possibility of this processing taking place. 
Therefore, it is believed that introducing the control of latency of stimulation will 
restore this behaviour to the auditory brain. In addition, there is evidence that the 
inhibitory pathways in the hearing impaired person are compromised because of 
5 lack of auditory input for some time. Therefore, the strategy may improve cochlear 
implant users' speech perception, especially in the presence of noise. 

The following description refers in more detail to the various features of the 
method and system for generating stimuli of the present invention. To facilitate an 
understanding of the invention, reference is made in the description to the 
10 accompanying drawings where the invention is illustrated in a preferred 
embodiment. It is to be understood however, that the invention is not limited to the 
preferred embodiment as shown in the drawings. 

In the drawings: 

Figure 1 is a schematic diagram of a first embodiment of a system for 
1 5 stimulating an electrode array implanted into a cochlea; 

Figure 2 is a schematic diagram of a second embodiment of a system for 
stimulating an electrode array implanted into an auditory brain; 

Figure 3 is a schematic diagram showing the function blocks of a processor 
forming part of the electrode stimulation systems shown in Figures 1 and 2; 

20 Figure 4 is an exemplary form of the latency function used to determine the 

amount of delay to be introduced for a particular electrode of the electrode 
stimulation system shown in Figures 1 and 2; and 

Figure 5 is a schematic diagram of an embodiment of a system for 
selectively releasing drugs in an array of drug delivery units and stimulating an 
25 electrode array implanted into a cochlea. 
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Referring now to Figure 1, there is shown generally a system for generating 
stimuli for application by an auditory prosthesis, including an array of stimulation 
devices, in response to a processed signal. In this embodiment, an electrode array 
1 for implanting into a cochlea connects via cable 2 to a receiver-stimulator unit 

5 (RSU) 3. The physical form of the electrode array may be different to that shown 
in the figure and depends on the device and location of implantation. The 
implanted system receives control signals and power from an external speech 
processor unit, preferably via a tuned coil RF system 5, 6 as illustrated. However, 
any alternative connection technique such as percutaneous connection may be 

10 employed or a fully implantable device may be used that does not require 
transmission through the skin. 

The coil 6 carries a signal modulated by the processor 7 so as to cause the 
RSU 3 to activate the electrodes in the electrode array by applying stimulation 
pulses in a desired sequence, timing and amplitude. The processor 7 in turn 
15 receives electrical analog signals from a microphone 8 worn by, or implanted in, 
the user. The present invention is concerned with the operation of the processor 
and particularly the method of post-processing the stimulation sequence for 
activating the electrodes. 

Figure 2 shows a variation to the system shown in Figure 1 in which the 
20 electrode array V forms part of an auditory brain implant that bypasses the 
cochlea altogether. The electrode array V is attached directly to the auditory brain 
at the base of the brain or some position higher up in the auditory pathway. The 
electrode array 1* forms a grid mapped to the form a linear electrode array. 

Figure 3 illustrates the various functional blocks of the processor 7, 
25 including a pre-filtering & ADC block 9, a filter bank 10, a stimulus generation 
block 11, a timing latency block 12, a stimulation selection & ordering block 13 and 
a loudness growth function block 14. The pre-filtering & ADC block 9 may be 
implemented using known electronic circuitry and analog signal sampling 
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techniques, whilst the functional blocks 10 to 14 may be implemented using known 
digital signal processing techniques. 

Base sound processing 

Sound is recorded by the microphone 8, which may inherently apply pre- 
5 emphasis to the incoming signal. This signal is low-pass filtered, to prevent 
aliasing during sampling, and is then sampled by an analog-to-digital converter in 
the pre-filtering and ADC block 9. The base sound processing is then performed 
on the signal that divides it up into a number of channels representing different 
frequencies 10. In an embodiment of the invention using the CI-24M cochlear 

10 implant, up to 22 channels may be used. The base sound processing system then 
determines the electrodes to be activated by application of stimulation pulses, and 
its own determination of the times that this activation should take place 11. For 
the Peak-Derived Timing Stimulation strategy, these are the times that each 
filtered waveform reaches a peak between zero crossings; for the Travelling Wave 

15 and STAR strategies, these are the times extracted from the rectified or threshold- 
crossing times of each filtered waveform plus the travelling wave delay introduced 
for each electrode. Other base stimulation strategies have their own method of 
generating the stimulation times. 

Introduction of amplitude-based excitation time latency 

20 Additional latency is introduced based on the amplitude of a filter band 

relative to the amplitudes of neighbouring filter bands in the filter bank 10. The 
function displayed in Figure 4 illustrates a possible form of the latency function 
utilising lateral temporal delay. This works in a manner similar to traditional lateral 
suppression, where the amplitudes of proximate frequency components suppress 

25 the amplitude of the filter band, except that instead of changing amplitudes of 
outputs, the timing of outputs is adjusted. Accordingly, the temporal adjustment 
applied to the activation time of each electrode is derived from the amplitudes of 
stimuli to be applied by proximate electrodes. In this figure, the stimulating 
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electrode represents an electrode which will be activated by application of a 
stimulus current pulse. 

The time of activation of the electrodes proximate to the stimulating 
electrode is determined by the following formula : 

5 T x = T hase (x) + f x (x) 

where T x is the time to stimulate electrode x, r base (x) is the base strategy's 

activation time for electrode x and f x (x) is the temporal delay caused by the 

relationship of the amplitude of this filter band to the surrounding filter bands for 
other electrodes. 

10 The latency function f x (x) defines a Mexican-hat shape centred on the 

stimuli to be applied by the proximate electrode, with the further restriction that it 
be limited to a minimum of zero, or no delay. A delay will be introduced if the 
weighted sum of the amplitudes of the filter bands for surrounding electrodes 
exceeds the value of the current electrode's amplitude. For an electrode that is a 
15 local maximum, the weighted sum will be negative and so no extra delay will be 
added. However, if there is a nearby electrode or group of electrodes with greater 
amplitude, then the activation time will be delayed. The formula for f x (x) is of the 
form 

N 

f x (x) = min(0,-2aA x + a^Tg(y)A y ) 

20 where A x is the amplitude of a stimulation to be applied by electrode x, a is a 

scaling factor, N is the number of electrodes to which the latency function is 
constrained, and g(y) is amplitude of a stimuli applied by each electrode y , as 
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illustrated in Figure 4. The latency function may be constrained to a limited number 
of electrodes only in the electrode array or may include all electrodes in the array. 
The actual value of N may vary according to the listener, the auditory prosthesis 
and the aural environment in question. 

5 The electrode array 1 shown in Figure 1 typically includes 22 electrodes, 

however the latency function is constrained to a limited number of electrodes 
proximate the electrode to which a high-amplitude stimuli is to be applied. In the 
example shown in Figure 4, the latency function is constrained to apply to 4 
electrodes at lower frequencies and five electrodes at higher frequencies (although 
10 four of these make no contribution). It will be appreciated that Figure 4 represents 
an exemplary subset of electrodes in a typical cochlear implant or auditory brain 
implant. Multi-electrode implants currently include from 4 to 22 electrodes although 
in future designs even more electrodes may be included. The invention may be 
applied to multi-electrode implants including any number of electrodes. 

15 Moreover, different extents and shapes of latency functions may be used in 

other embodiments of the invention, including different frequency extents on the 
low and high frequency sides and this may vary for different stimulating electrodes. 
It is to be understood that the latency function described above is merely one 
possible form of a latency function suitable for use with the present invention, and 

20 that other functions may be envisaged by a skilled addressee that cause the 
stimulation of electrodes representing lower-amplitude components of the signal to 
be delayed relative to stimulation of a proximate electrode representing a higher- 
amplitude component of the signal. 

A scheme for auditory brain implant processing is similar to that described 
25 above and illustrated in Figure 4. The electrodes are not usually in a linear 
configuration, so the scheme operates by using the frequencies that are assigned 
in the method usually prescribed for auditory brain prostheses and mapped to the 
form of the electrode array illustrated in Figure 4. 
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Further processing 

After the introduction of latency to electrode stimulation time, control passes 
back to the base stimulation strategy. It is at this stage that electrode selection and 
ordering must take place to determine which electrodes will be activated over the 
5 coming time interval and to deal with issues of contention where the activation 
times for two or more electrodes may be the same in the stimulus selection and 
ordering block 13. Implants are limited in the number of stimuli that may be 
provided per second, so electrode selection is required to ensure that this limit is 
observed. The procedure for doing this is usually to choose those electrodes with 
1 0 the largest amplitude of stimulation, but this is dependent on the base strategy and 
the prosthesis that is being used. 

Most current implants require that activation of multiple electrodes be non- 
simultaneous. It is for this reason that contention may be considered. This is also 
dependent on the base strategy, but where the base strategy does not consider 

15 this possibility, low amplitude stimuli may be discarded or delayed further in 
preference for higher amplitude stimuli. The stimulation provided by an auditory 
prosthesis that requires non-simultaneous activation may be divided into 
stimulation slots, where there is one slot for each activation time permitted. For 
example, the CI-24M implant, which has a maximum stimulation rate of 14,400 

20 pulses per second, has 14,400 stimulation slots per second. Stimuli are assigned 
to these stimulation slots by the base strategy. The extra delay to be introduced to 
overcome electrode contention may be performed by shifting the lower amplitude 
stimuli into the next slot. Then the next slot is considered and if multiple stimuli are 
in that slot, then the lower amplitude stimuli are shifted to the next slot. This 

25 procedure continues for each stimulation slot. If a stimulus has been postponed for 
more than a reasonable amount of time, then it will be discarded to avoid 
extraneous stimulation. This time could be around 0.5 msec (corresponding to 
about 22 stimulation slots for the CI-24M) although it may vary for different 
electrodes and different implementations. 



WO 2005/057983 



14 



PCT/AU2004/001729 



After each cycle of electrode selection, the stimuli are mapped to current 
levels using the standard loudness growth function (LGF) and the stored map (T 
and C levels) for the user by the loudness growth function block 14. The LGF is a 
logarithmic function relating stimulus level to loudness to obtain an appropriate 
5 increase in subjective loudness. The stored map specifies the minimum and 
maximum current levels permitted for a user. This is, again, a property of the 
implant and base strategy. 

The stimulus sequence is then transmitted to the receiver-stimulus unit 3 
that interfaces with the auditory prosthesis and encodes the electrode selection 
10 and current level information to the device. 

An alternative auditory prosthesis to which the present invention is able to 
be applied is a drug delivery neural implant array. This is a new form of implant 
that uses drug delivery arrays that can establish chemical interfaces with neurons. 
These arrays may also include electrodes for the activation of neurons by 

15 electrical current. Arrays that include both drug delivery devices and electrical 
stimulation devices are called hybrid neural implant systems. Drug delivery arrays 
selectively release drugs through drug delivery units, such as fluidic microchannels 
that activate receptors on discrete clusters of neural elements to stimulate neural 
activity. The delivery of drugs acts to alter the membrane potential of neurons and 

20 thereby excite or inhibit local neurons. An advantage of using drug delivery for 
neural stimulation is that an array will not stimulate the fibres of passage or 
activate backpropagation in neurons. 

Figure 5 shows an embodiment of a system for generating stimuli for application 
by an auditory prosthesis, including an array of stimulation devices, in response to 
25 a processed signal that is similar to the auditory prosthesis stimulation systems 
shown in Figures 1 and 2. In this embodiment however, the receiver-stimulus unit 
3- includes a drug supply 20 and pump 21 for delivering drugs stored in the drug 
supply device 20 to the usr via fluidic microchannels 22 running within the cable 2. 
The ends (the dark dots, an exemplary one of which is referenced 23) of the fluidic 
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microchannels form part of a hybrid array 1 — for stimulating the auditory brain. 
The hybrid array 1 — also includes electrodes (clear dots, an exemplary one of 
which is referenced 24) that are selectively activated by application of stimulation 
pulses via control circuitry 25 forming part of the receiver-stimulus unit 3-. In this 
5 case, the amplitude of the stimuli applied by the fluidic microchannels corresponds 
to the quantity of drugs delivered by each fluidic microchannel, whereas the 
amplitude of the stimulator applied by electrodes is determined by the amplitude of 
the electrical stimulation pulse applied to each electrode. The previously 
described method of generating stimuli for each stimulation device within the 
10 hybrid array 1- - conforms to the method previously described in relation to the 
embodiments of the invention shown in Figures 1 and 2. 

It is to be understood that various modifications and/or additions may be 
made to the method for modifying the latency of electrical stimuli described herein 
without departing from the spirit or ambit of the present invention. For example, in 
15 other embodiments of the invention, it is possible that the array of stimulation 
devices includes only drug delivery units so that stimulation of the auditory brain 
occurs only by means of the application of drugs without separate electrical 
stimulation. 



